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ABSTRACT 



The engagement of young Australians with school and 
individual and school-level influences on student engagement (defined as 
participation in extracurricular activities) were examined by analyzing data 
from the Longitudinal Surveys of Australian Youth. Being female, being from a 
higher socioeconomic background, and having professional parents were the 
individual-level factors associated with the highest levels of engagement 
with school. Students from independent schools had higher levels of 
engagement than did those in Catholic schools, who were more engaged than 
those in government schools. Students who planned on enrolling in tertiary 
study were more highly engaged than those who planned to leave school and go 
to work, and students at single-sex schools were more highly engaged than 
those at coeducational schools. Levels of engagement were higher where 
students believed that their school had a good school climate. Between-school 
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differences accounted for almost 9% of the variation in students* engagement 
levels, thus indicating that what school a student attends does matter. 
School administrators can influence student engagement by providing 
extracurricular activities and encouraging (or expecting) student 
participation. The following items are appended: descriptions of the samples 
and populations; descriptions of the individual-level variables; the factor 
score regression weights; the constructed student and school-level measures; 
and multilevel models. (Contains 17 tables and 36 references.) (MN) 
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EXECUTIVE SUMMARY 



This report examines the engagement of young people with school. Engagement is an important 
outcome of schooling, and there are a number of ways in which it can be defined. In this report, 
engagement is defined through Finn’s (1989) taxonomy of engagement or participatory 
behaviours, which examines students’ level of participation in the extracurricular activities 
offered to them by their schools. Finn (1989) argued that with such participation comes 
identification with school, a ‘belonging’ that can help to promote a feeling of self-worth and 
assist students to become resilient learners, particularly if they are part of a group at risk of 
leaving school before completing Year 12. Participation in extracurricular activities has been 
described as providing all students with an educational safety net, and several US studies have 
found participation to be positively related to a range of positive educational outcomes. In a 
British study utilising a large national database of student and parent attitudes to school. Barber 
(1996) found that students’ enthusiasm for extracurricular activities was seen by them as a 
particularly positive aspect of the school experience, and that parents strongly endorsed their 
value. An enduring effect of extracurricular participation has been shown to be the greater 
likelihood of participating in voluntary social activities as young adults (Lindsay, 1984). 

The study is based on students who were in Year 10 in 1999 using data from the 1998 cohort of 
the Longitudinal Surveys of Australian Youth (LSAY) project to investigate school-level 
differences in levels of engagement. It makes use of the full range of data for this particular 
cohort, analysing data from student, teacher and school-level questionnaires. 



The relationships between engagement and student and school-level factors 

The first analytical part of the report examines the relationship between a students’ sense of 
engagement and a variety of demographic, socioeconomic, educational and psychological factors, 
both at the individual level and at the school level. The major findings were: 

• Females had higher engagement levels than males. This was apparent in all school 
sectors and at all achievement levels; 

• Students from higher socioeconomic backgrounds and those with professional parents 
had the highest levels of engagement with school; 

• Students from independent schools had higher levels of engagement than those in 
Catholic schools, who in turn were more engaged than those in government schools; 

• Students who plan on enrolling in tertiary study were more highly engaged than those 
who planned to leave school and go to work; 

• Students at single-sex schools were more highly engaged than those at co-educational 
schools; 

• Levels of engagement were found to be higher where students believed that their school 
had a good school climate, that is one where they have high quality teachers, effective 
discipline, high levels of student learning and a positive school spirit; 

• Students who were generally happy with school and with learning (as measured by the 
positive affect scale) were more engaged than those who were not; and 

• Students who were intrinsically motivated (e.g. I find school work interesting, I like to do 
my best) were found to be more engaged than those who were not so intrinsically 
motivated. 
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School and individual effects on engagement 

The second analytical part of the report examines whether there are significant between-schools 
differences in engagement and second, after finding that there are, which particular student and 
school characteristics lift engagement, taking into account differences in the academic and social 
mix of students across schools. This was carried out by fitting a series of multilevel models, with 
students at the first level and schools at the second level. Multilevel modelling (or multilevel 
regression) disaggregates the relationships between engagement and its predictors into a within- 
school and a between-school component, which allows us to examine the effects that schools 
have on engagement levels. For example, a student fi'om a low socioeconomic background might 
be predicted to be less engaged than a student fi'om a high socioeconomic background who 
attended the same school. However some other school might be so effective at promoting 
engagement that a low socioeconomic background student would have a higher predicted level of 
engagement than the high socioeconomic status student in the first school. 



This report uses an aggregated measure of student engagement, referred to as school-level 
engagement, as a predictor in the multilevel modelling. While this variable was derived from the 
student-level engagement measure, at the school-level, it becomes a proxy, contextual measure of 
a school’s normative environment. This variable represents, as best we are able to measure it 
with these data, a school ‘ethos’ of participation. It is an average for a school, and as such, there 
are deviations fiom the mean that would provide a starting point for further qualitative research 
studies. 

The major findings fiom the multivariate, multilevel investigation of the role of schools and the 
influence of individual-level factors on student engagement are; 

• Between-school differences account for almost 9 per cent of the variation in students’ 
engagement levels. While this is not large, it is significant, and indicates that it does 
matter what school a child attends; 

• The overall level of student engagement in the school was a strong predictor of student- 
level engagement. High engagement at the school level (which measures the effect of 
engagement over and above that at the personal level) was found to moderate the 
negative effects of socioeconomic status and indigenous status. This finding indicates 
that the school environment has an important influence on student engagement; 

• Gender, parents’ educational level, student perceptions of school climate, self-concept of 
ability and intrinsic motivation were all found to have an effect on individual 
engagement, over and above the influence of whole-school engagement. The effect of 
gender, in that females were much more highly engaged than males, was next strongest to 
the influence for whole-school engagement, while the other influences were much 
weaker; and 

• Different predictors were found when the analysis was conducted separately for males 
and females. Apart fiom whole-school engagement, which was the strongest predictor 
for both males and females, the analysis found that parents’ educational level and student 
perceptions of school and class climate were the strongest predictors for males. For 
females, the strongest predictors were found to be socioeconomic status, self-concept of 
ability, perceptions of school (but not class) climate, and attendance at a coeducational 
school. 



These findings are important firstly fiom a policy perspective. Student engagement through 
participation in extracurricular activities is able to be influenced by school administrations 
through policies on the provision of extracurricular activities and explicit encouragement (or even 
an expectation) of student participation in such activities. 
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Secondly, the findings are important because they isolate several differences in factors affecting 
the engagement of males and females that can be influenced by schools. For males, attention in 
schools needs to be paid to classroom and school climate. Males appear to need more of a 
supportive school and classroom environment to be engaged with their school. They need to be 
strongly encouraged by their schools and by their parents to participate in extracurricular 
activities, and a broader range of activities developed by schools that are appealing to young 
males. 

For females, schools need to focus on developing a strong self-concept of ability and positive 
views of school climate. Whilst for males, parents’ educational level, and for females, 
socioeconomic status, are not malleable, their effects are small compared to the effects of overall 
high levels of school engagement. 

This report finds that it does matter which school a student attends. Students at schools which 
have the resources or the commitment to provide a broad range of extracurricular activities and 
encourage students to participate, generally have higher levels of engagement than those in 
schools which do not. Strong participation in such activities leads to a student’s closer 
connectedness to the school community, and it is argued in the report that there are ‘flow-on’ 
effects to more academic parts of the curriculum. As an equity issue, it is important that this be 
addressed. Socioeconomic status is a persistent influence on participation, both at the individual 
level and at the school level. At present, students with parents who have the financial resources 
to allow a wide participation in extracurricular activities obtain a benefit from schooling that 
those students vrith less access to financial resources do not. 
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Student engagement with school: Individual and school-level influences 



1. INTRODUCTION 



Student engagement with the intellectual work of schools is a primaiy goal of education, however 
taking achievement as the only outcome of 13 years of education is a narrow focus, and there is 
strong evidence that the Australian school community has a much broader view of the central 
purpose of schools. For example McGaw, Piper, Banks and Evans (1992) reported that 
Australian school communities expressed a belief that one of the key outcomes of schooling was 
the development of students’ personal and social skills, as well as positive self-concept, self- 
discipline and self-worth. Extracurricular activities may provide an opportunity for students to 
develop these skills in a less formal setting than the classroom. As well, students who develop 
positive relationships with school, through the classroom or through participation in 
extracurricular activities, are more likely to become lifelong learners, moving in and out of 
education and training as they find it necessaiy. In the short term, they are more likely to remain 
at school to complete Year 12, whatever their achievement level (Holland & Andre, 1987; 
Mahoney & Cairns, 1997; Marks, Fleming, Long, & McMillan, 2000). To understand how to 
assist students to develop this connectedness to school, research needs to focus not just on 
students’ test scores, which puts the focus on what is learnt, but on what it takes to develop the 
learner. This particular piece of research examines the influences on students’ participation in 
extracurricular activities. 

Across Australia, students have the opportunity to participate in an array of extracurricular 
activities organised by their schools. These include sports programs, community programs, 
school governance, music, art and drama, academic and vocational clubs as well as programs 
such as peer mediation and peer support. These extracurricular activities are recognised widely 
by teachers and parents as providing students with opportunities for leadership, for personal 
growth and for developing a sense of commitment to the well being of the wider community. 
They offer students opportunities to apply skills learned in the classroom in an applied setting, for 
them to learn the value of teamwork, competition and cooperation, individual and group 
responsibility. 

This study draws on Finn’s (1989) identification of participatoiy behaviours as a measure of 
students’ engagement with school. Finn (1989) argued that young people’s level of autonomy 
and their opportunity for participation in extracurricular activities increases with age, and that 
many students participate in such activities both in addition to, and sometimes instead of, the 
formal curriculum. Participation in such activities promotes a sense of engagement with school, 
and the current report examines the extent to which this form of engagement is influenced by a 
student’s personal characteristics, or by the school that they attend. 

Most of the research on school effectiveness has focused on outcomes in terms of academic 
achievement; less attention has been paid to how well schools engage students in learning and in 
school life and how this affects students’ outlooks on school and the future. However the 
purposes of schooling are varied. An academic perspective of schooling focuses on intellectual 
competence, and emphasises the purpose of schools as transmitters of formal knowledge and the 
pursuit of academic excellence. In contrast, a developmental perspective emphasises the 
provision of a variety of experiences to ensure the psychological, social and physical 
development of all individuals in a school. These perspectives are not contradictoiy, and schools 
need to address engagement issues as well as those of achievement. For instance, summarising 
their report on what makes schools effective, McGaw et al. (1992) concluded that: 

School effectiveness is about a great deal more than maximising academic 

achievement. Learning and the love of learning; personal development and self- 
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esteem; life skills, problem solving and learning how to learn; the development of 
independent thinkers and well-rounded confident individuals; all rank as highly or 
more highly as the outcomes of effective schooling as success in a narrow range of 
academic disciplines, (p. 174) 

At Year 9 level for this particular cohort, around 80 per cent of students are certain that they want 
to remain at school until Year 12. While this aspiration may ebb and flow somewhat over the 
next three years of their schooling, the apparent annual retention rate to Year 12 nationally 
remains only at about 72 per cent (Australian Bureau of Statistics, 2001). In many cases students 
leave against the express advice of their parents and teachers, and one of the major reasons that 
they leave is because of disengagement with school and with learning. It is likely that some 
groups of students are more engaged than others, and that some schools are more effective than 
others at engaging students. This report examines some of the contributing factors to these 
differences. 



Defining and measuring Engagement 

There are a number of facets to students’ engagement with school. One is engagement with 
learning, another is engagement with the school community. While this study only examines the 
latter, it is likely that there is a synergistic relationship between the two facets. The study draws 
on Finn’s (1989) identification of participatory behaviours as a measure of students’ engagement 
with school. While Finn’s taxonomy comprises four levels, it is the third level only that is 
examined in this study. Finn’s third level of engagement is defined as ‘‘participation in the social, 
extracurricular, and athletic aspects of school life in addition to or in place of extensive 
participation in academic work” (Finn & Rock, 1997, p, 222), 

Finn (1989) argued that students who regularly participate in extracurricular activities develop a 
sense of ‘belonging’ to their school commxmity, in that they are a conspicuous part of the school 
and that the school is an important part of their own lives. This ‘belonging’ can help to promote a 
feeling of self-worth and assist students to become resilient learners, particularly if they are part 
of an “at-risk” group in terms of school completion (Mahoney & Cairns, 1997) and other 
educational outcomes. Mahoney and Cairns (1997) found not only that engagement in 
extracurricular activities was linked to decreasing dropout rates, but that this outcome was 
strongest for those students who were at highest risk of dropping out. They hypothesised that for 
those students v^th marginal attachment to school and its values, participation in extracurricular 
activities “provides an opportunity to create a positive and voluntary connection to the 
educational institution ... unlike alternative procedures which focus on the deficits of students” 
(p. 248). 

While there are few studies examining the relationship of participation in extracurricular 
activities with academic achievement in school, Holland and Andre’s (1987) review of research 
in this area concluded that participation is correlated with a number of desirable outcomes, 
including higher levels of self-esteem and feelings of control over one’s life, higher educational 
aspirations and higher grades, especially among males. 

Participation has effects because of what happens as a result of participation ... 

Participation may lead students to acquire new skills (organizational, planning, time- 

management etc.), to develop or strengthen particular attitudes (discipline, 

motivation), or to receive social rewards that influence personality characteristics. 

(Holland & Andre, 1987, p. 447) 

Using High School and Beyond longitudinal data. Camp (1990) examined the effects of extent of 
participation in extracurricular activities on student achievement, controlling for the effects of 
other variables such as socioeconomic status and prior academic achievement, using structural 
covariance analysis. His study found that students’ participation levels produced a significant 
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positive effect on achievement, an effect found to be more than twice the size of student study 
habits. 

Marsh’s (1992) analysis of the same database, focussing on the effects of the amount of 
extracurricular participation, found that the significant effects of total extracurricular activity 
participation were small but consistently positive. Marsh (1992) found that the total amount of 
activity typically facilitated academic outcomes rather than detracted from them, and that these 
positive effects were generalisable across a variety of student backgrounds. He concluded that 
‘‘participation in extracurricular activities - even those not obviously associated with academic 
achievement - apparently leads to increased co mmi tment to school and school values, which 
leads indirectly to increased academic success” (p. 560). Gerber’s (1996) examination of the 
relationship between extracurricular participation and academic achievement using National 
Educational Longitudinal Study (NELS) data similarly found a positive relationship between 
participation levels and academic achievement, and the author argued that participation “may be 
one important source of identification with school, especially for academically weak students and 
those who are at-risk for withdrawing” (p. 50). 

In a UK study. Barber (1997) examined a small number of schools cited by OFSTED (Office For 
Standards in Education) as being worthy of special recognition in the annual report of the Chief 
Inspector, comparing extracurricular provision to those in a similar number of non-cited schools. 
While the results of such an examination cannot be generalised, it was found that extracurricular 
provision and levels of participation in extracurricular activities was much higher in these 
exemplar schools. Barber (1996) also provides evidence from the Keele University national 
database on student and parent attitudes to secondaiy school that students are enthusiastic about 
extracurricular participation and that parents see it as an important aspect of schooling. 

An enduring benefit of extracurricular participation was found to be higher participation rates in 
voluntaiy social activities as a young adult (Lindsay, 1984). This study used a large national 
longitudinal database in the US to examine the effects of extracurricular participation on both 
educational attainment and on social participation as a young adult, accounting for the effects of 
prior academic attainment, gender, socioeconomic status, and sociability. The author found that 
extracurricular participation did indeed affect participation, controlling for all other variables, and 
that this effect was a somewhat more powerful influence than educational attainment. 

There are a number of other benefits of students’ engagement with school. For example, young 
people who have positive feelings towards school and who are active participants in a variety of 
school activities are more likely to stay in school and are more likely to become independent 
learners (Ainley, Batten, & Miller, 1984; Ainley, Foreman, & Sheret, 1991; Ainley & Sheret, 
1992; McNeal, 1995). Other studies have found positive relationships between a student’s 
engagement and academic achievement and with other educational outcomes, including better 
attendance and aspirations to higher levels of education (Connell, Spencer, & Aber, 1994; Firm & 
Rock, 1997; Marsh, 1992; National Centre for Education Statistics, 1995), while dissatisfaction 
with aspects of school life has been demonstrated to be a key issue for non-completion of 
secondaiy school (Batten & Russell, 1995; Holden & Dwyer, 1992). 

In more recent analyses of the LSAY data, Marks, Fleming, Long and McMillan (2000) found 
that a variety of factors, such as level of engagement in school life (measured by participation in 
extracurricular activities), academic self-concept, educational aspirations and parental 
expectations were significant influences on Year 12 participation. Lamb, Dwyer and Wyn (2000) 
cited negative experiences of schooling or poor academic self-concept as reasons given by many 
students for not completing school. 

It is clear from the research that engagement is an important construct, both in terms of its effect 
on a variety of other educational outcomes and because it is a valuable outcome in itself. 
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However Finn’s (1993) taxonomy of engagement is largely a function of the individual, and it is 
difficult to find studies that have attempted to link characteristics of schools that reach beyond the 
classroom v^th student’s engagement. The model analysed in this report includes school-level 
influences such as the organisational environments in which these students are situated. 



Key Questions 

A number of key questions were developed to guide the data analysis. 

• What is the influence of socioeconomic background on student’s engagement levels? 

• What is the influence of gender on student’s engagement levels? 

• To what extent can differences in student engagement be attributed to differences between 
schools? (The variation between schools in student engagement is an indication of the 
influence of individual schools. If the between-school variance is large, then it is clearly 
important which school a student attends. If the between-school variation is small then the 
school a student attends makes little difference to their engagement.) 

• What are the characteristics of students or schools that enhance a student’s engagement 
with school? 



Organisation of the report 

This report comprises five chapters. Chapter 2 of this report will examine the variables and 
methods used in this report, including a brief account of multilevel modelling and its particular 
use in this study. 

Chapter 3 provides a univariate analysis of the student-level and school-level variables that are 
included in the theoretical model proposed, as well as their relationship with the outcome 
variable, student engagement. 

Chapter 4 investigates the influence of schools and school characteristics on student engagement 
with multilevel analysis, adjusting for differences in the home background of the student. 

Chapter 5 discusses the results of the study and policy implications of the findings. 




U 



2 . 



DATA AND METHODS OF ANALYSIS 



This chapter provides the reader with information to facilitate their understanding of the results 
presented in tables and discussed in the text of the following chapters. The initial section provides 
a description of the LSAY data used in the report. 



Data 

In 1998, a nationally representative sample of 14,1 18 Year 9 students was selected to form the 
second cohort of the new program of the Longitudinal Surveys of Australian Youth. The initial 
sample was constructed by randomly selecting two to three Year 9 classes from a sample of 
schools designed to represent State and sector. Smaller States and Territories were over-sampled 
to provide sufficient numbers to provide reliable State estimates on a number of key variables. 
This sample, termed the Y98 cohort, was surveyed again in 1999, this time with a mail survey. 
Attrition for this survey resulted in there being 1 1,150 cases available for analysis. More details 
about the sample can be found in Appendix 1 . 

Early in 1999, two questionnaires were sent to the participating schools. One of these was a 
School Questionnaire, which was completed by the Principal or a representative; the other was a 
Teacher Questionnaire, which was completed by a sample of ten Year 10 teachers. The school 
questionnaire collected information on school programs, school organisation and timetables. The 
teacher survey included questions on aspects of teaching and learning, as well as teachers’ levels 
of satisfaction with a variety of aspects of schooling. Because of the limitations of the sampling 
techniques (in that these teachers were not necessarily those of the students surveyed), the teacher 
data was averaged to school-level, to provide variables at the school level that would represent an 
overall view of a number of facets of school climate, environment or ethos. Personal 
characteristics of teachers such as gender, experience teaching, and qualifications could not be 
used in this analysis, as they could not be meaningfully averaged to school-level. 

Data were obtained from 1,716 teachers and from 218 schools of the 300 schools in the initial 
sample. Not all of the schools that provided a School Questionnaire also provided Teacher 
Questionnaires and vice versa. 



Construct measurement 

Most theories and models in educational research are formulated in terms of hypothetical 
constructs or latent traits that are not directly measurable. In survey research, it is usual to 
compute complex composite variables from responses to several questionnaire items measured on 
dichotomous or Likert-type ordinal scales. For this report, items were first selected to form 
composite variables a priori, based on substantive grounds. Exploratory factor analysis was then 
used to confirm the single factor structure of each set of items. 

The resultant composite measure has been traditionally computed either as a sum of factor scores 
or as simple, unit-weighted additive indices of their indicators, regardless of either the 
measurement or distributional properties of the constituent variables or of their relative 
contribution to the composite scale. These indices are then usually treated as continuous 
variables for the purposes of general linear model techniques, which assume that there is no 
measurement error associated with these indices. 

However Rowe (2000) warned that this approach ignores the possibility that some factors may 
contribute more than others to a scale and ^at the resulting composite scale may be invalidated if 
one or more of the indicator variables measure a latent trait other than the one under 
consideration. 
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The composite variables used in this report were constructed to take into account the different 
weights of the constituent items and measurement error. The method used was confirmatory 
factor analysis (CFA) using LISREL 8.3 (Joreskog & Sorbom, 1999) under a weighted least 
squares method of estimation and a listwise deletion of missing data. These composite variables 
were obtained by fitting one-factor congeneric models to the ordinal-scaled item data, based on a 
scaled covariance matrix of the polychoric (ordinal with ordinal) correlations using PRELIS 2 
(Joreskog & Sorbom, 1999). It is argued that the use of structurally sound and reliable composite 
scores is crucial in fitting multilevel regression models (Bryk & Raudenbush, 1992; Goldstein, 
1995; Rowe, 2000). Unlike traditional unit-weighted methods for computing composites, the use 
of factor score regression weights obtained from one-factor CFA models minimises measurement 
error in the items contributing to each scale, thus increasing the reliability (and validity) of the 
computed scale scores. The factor score regression coefficients for all constructed scales are 
provided in Appendix 3. 

Deflning and measuring Engagement 

As was discussed earlier. Engagement in the LSAY data is defined in terms of extracurricular 
participation. Several items were framed for the questionnaire administered to students when 
they were in Year 10 to identify this level of participation. The item is headed “Extra activities 
organised by school”, and to ensure that students identify extracurricular activities rather than 
those which are provided as part of the curriculum, the explanation continues: 

“^5 well as the subjects they teach, schools organise many different out-of-class activities for 
students. How often do you take part in the following school-organised activities ? '' 

• Sport 

• Music, band or orchestra 

• Debating 

• Drama, theatre, dance or school play 

• Community and support work at school (eg peer support, fundraising). 

Resposes to these items were re-coded from the original to a five-point scale with 0 representing 
no participation in the activity through to 4 representing participation in the activity on at least a 
weekly basis. The one-factor model is illustrated diagrammatically in Figure 1, which shows the 
latent variable Engagement on the right and the measured variables in rectangles on the left. 




Figure 1 One factor congeneric measurement model for Engagement 
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Goodness-of-fit statistics indicate that this model is a good fit to the data. The Root Mean Square 
Error of Approximation (RMSEA) was .0433*, while the Adjusted Goodness of Fit Index (AGFI) 
was .994, indicating that the five factors accounted for around 99 per cent of the variance in 
Engagement. If we were developing the items and trialing them we might re-consider the use of 
the items relating to participation in sport and in community and support work, as these items add 
little to the model. In this case however, it is more expedient to use all of the data that we have 
available, and to weight the items proportionally by their actual contribution to the scale. 



Other constructed variables 

The techniques described in the previous section were used in the construction of each composite 
variable that was used in this report. At the student level, variables were constructed representing 
students’ views of their classroom and school climate. The Teacher questionnaire has not been 
used in any previous LSAY reports. Factors were identified a priori, and exploratory factor 
analyses supported the hypothesised groupings. Confirmatory factor analysis was then carried 
out to derive the appropriate weights for each item on the scale. All of the factor score regression 
weights that were used are detailed in Table A1 in Appendix 3. The composite variables that 
were used are detailed in Appendix 4. 

All variables used in multilevel modelling were normalised so that they were directly comparable 
in size. 



Influences on engagement 

Figure 2 provides a fi’amework for this investigation. While many factors can be hypothesised to 
influence student engagement as measured in this report, including school ethos, peer group, 
student background and student inclination, the analysis in this report is linuted to the data 
available fi’om the LSAY surveys. This section of the report presents a brief description of all 
measures used, other than those which are obvious, such as gender, school sector, and indigenous 
status. For a more detailed description of all factors the reader is referred to Appendices 2 and 4. 

Student-level variables 

Socioeconomic status was based on the male parent’s occupation and coded on the ANU3 scale 
of occupational prestige. 

Parent’s educational level was constructed fi-om the highest level of education of the female 
parent. 

Home language was based on the primary language spoken at home. 

Parents ’ country of birth was based on the male parent’s country of birth. 

The students ’ aspirations measure was based on student’s post-school educational plans. 
Achievement was based on students’ scores on literacy and numeracy tests completed when the 
students were in Year 9. 

Perceived class climate was a scale on which students were asked the extent to which students in 
their classes were eager to learn, made good progress, work hard and were well behaved. 

Perceived school climate was a scale on which students were asked the extent to which to rate 
their school on quality of teachers, effective discipline, student learning and school spirit. 

Self-concept of ability measured students’ perceptions of their ability in English, mathematics and 
overall. 



' The Root Mean Square Error of Approximation (RMSEA) allows us to calculate the probability of 
obtaining the same results if a similar sample was taken from the ‘super population’. In this case the 
probability would be (100 - 4.33) ~ 96%. 
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Quality of School Life - Positive Affect based on students’ perceptions of their own well-being. 

Quality of School Life -Intrinsic motivation was constructed from seven items which address 
student’s self-motivation in learning. 

Quality of School Life -Opportunity measured student’s belief in the relevance of learning. 
Quality of School Life -Achievement measured student’s sense of success in school work. 



School-level variables 

Classroom climate was a scale on which teachers were asked the extent to which students in their 
classes were eager to learn, made good progress, work hard and were well behaved. 

School climate was a scale on which teachers were asked the extent to which students in their 
school were eager to learn, made good progress, work hard and were well behaved. 

School problems assessed the frequency of occurrence of problems such as student absenteeism, 
vandalism, verbal abuse of teachers and poor student behaviour in the school. 

Parental involvement asked teachers to rate the perceived level of parental involvement with 
school decision making, camps and excursions, parent-teacher evenings and general 
support of the school’s goals. 

Satisfaction with workload measured teachers’ level of satisfaction with their workload, out-of- 
class duties, amount of committee work and class sizes. 

Satisfaction with resources measured teachers’ level of satisfaction with access to teaching 
resources, buildings and facilities, teaching equipment and grounds and sports facilities. 

Satisfaction with management measured teachers’ level of satisfaction with support from and 
style of management, and decision making processes. 

Relationships with others assessed the strength of relationships between teachers, between 
teachers and students and between teachers and parents. 

Academic success measured teachers’ perceptions of the level of success in their school in 
promoting academic success, such as in achieving good academic results, preparing 
students for higher education, and providing a broad range of co-curricular activities. 

Broad success measured teachers’ perceptions of the level of success in their school in promoting 
broad success, such as in providing a curriculum that caters for all students, teaching skills 
useful in employment and meeting the needs of disadvantaged groups. 

Rating teachers teachers to rate their colleagues, for example on mastery of their subject 
matter, communication, maintaining interest and managing discipline. 

Gender mix of school refers to whether the student attended a single-sex or coeducational school. 
Size of school is a continuous measure that refers to the number of students enrolled at the school. 

Student background and contextual factors including gender, socioeconomic status, indigenous 
status, language at home, aspirations and parents’ education, were hypothesised to affect a 
student’s level of engagement with school. A group of student-level attitudinal and achievement 
factors were also hypothesised to impact on student engagement with school. These included 
students’ self-concept of ability, level of achievement, beliefs about the quality of school life and 
perceptions of classroom and school climate. These factors are also influenced by social 
background, however this influence is not measured in this report. 

At the second level of the model are a range of teacher and school-level variables, which were 
hypothesised to have an independent effect on student-level engagement. Measured directly at 
the school level were factors such as school sector, gender-mix of school and school size. School 
size was included as a school-level variable as it has been shown to affect engagement with 
schooling (Lee & Smith, 1995, 1997). A measure of school-level engagement was also added at 
this level, and while this variable was derived from the student-level engagement measure, at the 
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school-level, engagement becomes a proxy, contextual measure of a school’s normative 
environment. Clearly, the average engagement of students within a given school (and its 
variance) may have an effect on students above and beyond the effect of an individual’s 
engagement. Outliers can affect a school-level average, and if, for educational policy reasons, we 
are interested in improving educational outcomes for students, then the use of such an average in 
this model allows us to identify those students who do not fit the pattern. That is, students with 
low levels of engagement in schools with high levels of engagement, and student vsdth high levels 
of engagement in schools with low levels. 

A number of variables derived from the teacher questionnaire were aggregated to school-level. 
Teachers are a socially diverse group; they come from differing ethnic and socioeconomic 
backgrounds and have differing work histories and academic credentials. It is reasonable to 
assume that this varied background creates diversity in the expectations and manner in which 
teachers experience the school, thus producing differing perceptions of the climate of the schools 
in which they work. By aggregating the teacher data to school level we create a proxy measure 
of overall school climate. 

The effects of these variables were examined using multilevel modelling techniques to account 
for the hierarchical structure of the data, that is students vsdthin schools. Failure to take into 
account this structure makes data analysis seriously prone to underestimation of standard errors 
and therefore erroneous conclusions. 

Multilevel modelling 

By its very nature, the investigation of school effects using the LSAY data requires the 
exploration of multilevel relationships. For LSAY, schools are first sampled and v^thin these 
schools, students within classrooms are sampled, and as a result of this method of sampling, the 
individual observations within schools are not independent. This dependence arises because of 
the shared experiences among students and/or teachers within a given school, and is due in part to 
the way in which persons are assigned to schools. For example, because samples of students or 
teachers within particular schools are, in general, more like each other than similar sampling units 
in other schools, to the extent that schools vaiy in their effects, the responses of students or 
teachers within a given school will be correlated. Further, since the distribution of student or 
teacher outcomes may vary as a flmction of school-level explanatory variables, these variables 
are parameterized by their partial regression coefficients within each school (see Burstein, Miller 
& Linn, 1981; Goldstein, 1987, 1995). This is due to the fact that the school-level variables are 
constant for students and/or teachers. 

As a result, the average correlation between variables measured on students from the same school 
will generally be higher than the average correlation between variables measured on students 
from different schools. Standard statistical tests rely on the assumption of independence of 
observations, and this assumption is generally violated in multilevel data, resulting in a spurious 
number of ‘significant’ results. Due to this, the means presented in Chapter 3 are not tested for 
significant differences, as the assumption of a simple random sample is violated and standard 
errors would be underestimated. 

Multilevel modelling (MLM) techniques are used to account for the inherent multilevel structure 
of the data. These techniques are essentially an extension of linear regression models, and 
assume that there is a single dependent variable measured at the lowest level and explanatoiy 
variables at all existing levels. In this study we utilise a two-level model, placing the student at 
the first level and the school at the second level. For further information related to this, the reader 
is directed to publications such as Bryk and Raudenbush (1992), Goldstein (1995), Kreft and de 
Leeuw (1998) and Snijders and Bosker(1999). 
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Before fitting the multilevel models, however, it is important to examine the distributional 
properties of the continuous variables to be used in subsequent explanatory modelling. For such 
purposes, PRELIS 2 (Joreskog & Sorbom, 1999) gives a detailed summary of the descriptive 
parameters and provides both univariate and multivariate tests of zero skewness and zero 
kurtosis^. The raw ‘composite’ scale scores were normalised using MLwiN (Rasbash, Browne, 
Healy, Cameron, & Charlton, 2000) for the multilevel analysis, as screening tests found 
significant levels of multivariate skewness and kurtosis. MLwiN (Rasbash et al., 2000) uses a 
method of re-scoring that assigns expected values fi*om the standard Normal distribution 
according to the ranks of the original scores in the form of Normal Equivalent Deviates. 

Graphical techniques 

Boxplots 

Boxplots are used to graphically illustrate the distributions of engagement scores in the sample, 
both within and across sociodemographic and educational groups. The horizontal line near the 
middle of a box represents the median value; that is, 50 per cent of the cases have engagement 
scores above this value and 50 per cent of the cases have engagement scores below this value. 
The top of the box represents the 25*** percentile; that is, 25 per cent of cases have scores above 
this value. Similarly, the bottom of the box represents the 75*** percentile, above which lie the 
scores of 75 per cent of students. The distance between the top and the bottom of the box is 
referred to as the inter-quartile range, which can be used as a summary measure of the ‘spread’ of 
engagement scores. The ‘whiskers’ above and below the box represent the 10*** and 90*** 
percentiles: 10 per cent of cases have values above the 10*** percentile; and 90 per cent of cases 
have values above the 90*** percentile. 



^ Skewness is a measure of symmetry, or more precisely, the lack of symmetry. A data set is symmetric if 
it looks the same to the left and right of the centre point. A skewed data set has many more values at 
one end than the other. Kurtosis is a measure of whether the data are peaked or flat relative to a normal 
distribution. That is, data sets with high kurtosis tend to have a distinct peak near the mean, decline 
rather rapidly, and have heavy tails. Data sets with low kurtosis tend to have a flat top near the mean 
rather than a sharp peak. 




20 



Student Engagement with School 



11 



School-level 



SCHOOL FACTORS 
School sector 
School type 
Size of school 
Mean level of engagement 

TEACHER FACTORS (SCHOOL LEVEL) 
perceived class climate 
perceived school climate 
school problems 
parental involvement 
satisfaction with 
o workload 
o resources 
o management 
o students 

relationships (between teachers & 

principal/parents/peers) 

success of school academically 

broad success of school (ie caters for all students) 

rating of other teachers 



Student-level 



STUDENT 

ENGAGEMENT WITH 
SCHOOL 

Participation in extra- 
curricular activities 



attitudes & 

ACHIEVEMENT 
Self-concept of ability 
Perceptions of class and 
school climate 
Quality of School Life 
Achievement 




STUDENT BACKGROUND & 
CONTEXTUAL FACTORS 
Gender 

Socioeconomic status 
Indigenous status 
Parents’ educational level 
Home language 
Student’s aspirations 
Parents’ birthplace 



Figure 2 Model proposed for explaining student engagement using LSAY data 
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3. STUDENT AND SCHOOL FACTORS AND THEBR RELATIONSHIP 

WITH ENGAGEMENT 

This chapter examines a variety of student-level and school-level factors that are hypothesised to 
influence engagement with schooling. The findings in this chapter are indicative only - no 
account is taken of the interrelationships between variables. 

Table 1 provides the means for engagement for each of the categorical student background 
variables identified: gender, indigenous status, achievement in literacy and numeracy, parents’ 
educational background, language background, school sector and type and student’s aspirations. 
Also provided is the proportion of students in each group who participated in each of the 
activities on a monthly or weekly basis. 



Table 1 Means on Engagement measure and percentage of students participating at least 
once a month, by student background factors 







Engagement 


Percentage of students participating at least 
once a month 




N* 


Mean 


SD 


Sport 


Music 


Debating 


Drama 


Comm- 

unity 


Gender 


















Male 


4122 


1 16 


“.78 


72 “ 


15 


1 


"ll 


” 19 


Female 


4274 


1.41 


.82 


62 


19 


8 


24 


28 


Indigenous status 


















Indigenous student 


186 


1.30 


.78 


' 77 


12 


10 


^ if 


25 


Non-indigenous student 


7905 


1.28 


.81 


67 


17 


7 


18 


24 


Parents’ educational level 


















University 


1731 


1.47 


.87 


72 


26' ' ' 


12 


20 


“'29'““'"“ 


TAFE 


1057 


1.36 


.82 


71 


16 


8 


21 


25 


Apprenticeship 


425 


1.25 


.79 


63 


15 


6 


17 


21 


Completed secondary school 


2015 


1.25 


.79 


67 


17 


6 


17 


23 


Did not complete secondary school 


2636 


1.18 


.76 


62 


13 


6 


16 


21 


Home langnage;^';'i;i^^ 


















English 


-^276“ 




.81 


66 




'"^"7'"' 


18“ 


23 


Non-English 


739 


1.33 


.80 


69 


18 


11 


16 


26 


Parent’s couniiy'drbirth ; 


















Australian 


' 5575 





.81 


.... 




7 


17 


23“ 


Other English-speaking country 


939 


1.28 


.82 


63 


19 


6 


19 


23 


Non-English speaking country 


1581 


1.33 


.80 


67 


17 


10 


16 


27 


Student’s aspirations : 

No post-school study 


1108 


1.05 


.74^ 


64 


10 


5 " 


12 


15“ 


Apprenticeship 


644 


1.10 


.72 


70 


10 


4 


13 


18 


Other or TAFE 


1536 


1.22 


.76 


66 


13 


5 


17 


22 


University 


4285 


1.40 


.84 


68 


22 


10 


20 


28 


School Sector 


















Government 


5549 


1.20 


.79 


63 


15 


6 


17 


20 


Catholic 


1748 


1.38 


.80 


73 


17 


11 


18 


31 


Independent 


1131 


1.55 


.85 


77 


31 


10 


19 


28 


School type 


















Coeducational 


4639 


125 


.81 


65 


17 


6 


18 


22 ■ 


Single-sex 


1217 


1.51 


.84 


77 


24 


14 


18 


32 


Early school achievement 


















Highest 


2188 


L39 


.85 


69“ 


24"' 


10 


16 


25"’ 


Upper middle 


2123 


1.31 


.82 


66 


19 


7 


20 


25 


Lower middle 


2097 


1.22 


.77 


65 


13 


7 


17 


22 


Lowest 


1987 


1.21 


.78 


68 


13 


6 


18 


21 



* Note: Ns sum differently due to varying student response rate 



o 

ERIC 



22 



Student Engagement with School 1 3 



Gender 



The means shown in Table 1 indicate that there are a number of gender differences in 
engagement levels, and that overall engagement levels are higher for females than males. In the 
context of this study, this means that female students are likely to participate to a greater extent in 
extracurricular activities than males, thereby increasing their level of attachment to the school. 
This finding is consistent with other research (Finn, 1989; Finn & Cox, 1992; Lee, Chen, & 
Smerdon, 1996; Lee & Smith, 1993, 1995). Males are more likely to participate in sports 
activities (which added little to the engagement model as shown in Figure 1), whereas females’ 
level of engagement is greatly enhanced by the much higher proportion who participate on a 
regular basis in drama and community work. 

Indigenotis status 

While it would appear fi^om these data that there is little difference in the level of engagement 
shown and by the participation levels in the activities by indigenous and non-indigenous students, 
some caution is required due to the low number of indigenous youth in the sample. 

Parents ' educational level 

There appears to be a positive linear relationship between engagement and parents’ educational 
level. Children of tertiary educated parents have higher levels of engagement than those whose 
parents attended a TAFE, and the lowest levels of engagement are seen for those young people 
whose parents did not complete secondary school. The differences are seen the most in the 
proportion of students who participate regularly in music and sport, while there are few 
differences in drama participation levels. 

Home language and parents ’ country of birth 

These two variables examine differences in level of engagement by ethnicity, however the 
differences between groups are quite small. In both cases there is a tendency for those with a 
language background other than English to have higher levels of engagement than those fi^om an 
English-speaking background or with parents bom in Australia. The participation rates show that 
the children of parents fi*om non-English speaking countries are more likely to participate in 
community support activities. 

Student 's aspirations 

Student’s aspirations should be associated with student engagement. It stands to reason that 
students who aim to complete secondary school and continue their education into the post- 
secondary sector would be more likely to be involved with their schooling and more likely to 
participate in extracurricular activities. Table 1 indicates that this is indeed the case, with levels 
of engagement being highest amongst those students who aim to continue their education and 
lowest levels amongst those who have no plans for post-school study. These students are a great 
deal more likely to participate in music, debating, drama and community work, with the 
differences being smallest in sport participation. 

School sector and school gender mix 

It would be hypothesised that students in the non-government school sector (encompassing both 
Catholic schools and non-Catholic independent schools) would have higher levels of engagement 
than those in the government school sector. In general, there is a much greater provision of 
extracurricular activities in the independent school sector, and in many schools it is compulsory 
for students (and indeed teachers) to participate. Those parents with students in the independent 
school sector are also more likely to have the financial resources to provide for their child to 
participate in a wide range of extracurricular activities. 
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These data demonstrate that students attending non-Catholic independent schools show higher 
levels of engagement than those in Catholic secondary schools, who in turn have higher levels of 
engagement than those in government schools. Similarly, students at single-sex schools have 
higher levels of engagement than those in coeducational schools. Participation in music is 
particularly high in independent schools, with almost one-third of students involved in music on a 
monthly or weekly basis, compared to one-sixth of students in government schools. Participation 
rates in drama activities are very similar in each school sector, and the same between 
coeducational and single-sex settings. 

It must be recognised, however, that there is likely to be confounding of these variables, in 
particular with parents’ education and socioeconomic status. Students from high socioeconomic 
backgrounds are more likely to be those with parents who have high levels of education, and are 
more likely to attend non-Catholic independent schools. As well, single-sex schools are more 
common in the Catholic and independent sectors. It is also more common for students in the non- 
government sector to be required to participate in extracurricular activities. It is very important 
that we examine differences in engagement, therefore, controlling for these background variables. 

Achievement level 

The mean levels of engagement as presented in Table 1 indicate that there are some differences in 
engagement according to achievement level. Engagement is higher among students with higher 
achievement levels. This concurs with previous findings that more academically successful 
students show higher levels of engagement with school (Lee & Smith, 1993, 1994). This is most 
likely to be due to the much greater involvement in music activities, with around one-quarter of 
students from the highest achievement level participating in music compared with one in eight of 
those from the lowest achievement level. 

Socioeconomic status 

As socioeconomic status in this report is measured with the ANUS occupational prestige scale, 
which is continuous, the analysis performed for the categorical variables is not appropriate; hence 
correlational analysis is presented in this section. The correlation of student engagement with 
socioeconomic status was found to be 0.11, which is small, but significant at the 10% level. As 
found for parents’ educational level, there is a positive linear relationship: students from higher 
socioeconomic levels show higher levels of engagement. 

In the next section of this report, some analyses will be reported that examine the extent to which 
several of the background factors might interact with gender in their influence on engagement. 
While there are many such possible combinations of interactions, gender was chosen specifically 
for the focus of this particular analysis. As has been previously discussed, gender is an important 
factor in engagement. A number of studies have reported differences in levels of engagement 
between males and females, in favour of females, and it is mooted that males’ alienation from 
school is becoming a major problem for the community. 



Interactions amongst background factors 

The previous section showed that school sector is associated with engagement. However is this 
association the same for males and females? Figure 3 indicates that while there is a similar 
association, there are some differences. 
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SEX 

l^ male 
I I female 



School sector 



Figure 3 Student engagement by school sector and gender 



Although the median score is higher for females than males within each school sector, and by a 
similar margin, the differences lie in the proportion of each of the distributions that overlap. This 
is smallest for the government schools, indicating that gender differences are greater within this 
sector. However the median score for engagement among females at government school is also 
higher than the median male score in any school system. 



Figure 4 shows the distribution of engagement by school type (single-sex or coeducational) and 
gender. Certainly females are more engaged within each different type of schooling, and 
females’ levels of engagement in coeducational schools are almost identical to the levels of 
engagement of males in single-sex schools. Gender differences are greater in coeducational 
schools. 




School type 



Figure 4 Student engagement by school type and gender 
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Achievement quartiles 



Figure 5 Student engagement by achievement and gender 



Figure 5 shows the distribution of engagement by achievement level and gender. At each level, 
females show higher levels of engagement than males, and while the differences appear to be 
largest at the highest achievement level these differences are probably not practically significant. 

Student attitudinal variables 

Table 2 provides the reader with correlations between the attitudinal variables and achievement. 
These variables were presented in Figure 2 as the second set of variables. It can be seen that 
there are positive (and significant, due to sample size) correlations with engagement for all of the 
attitudinal variables, as well as with the continuous measure of achievement. While these 
correlations are not large, they indicate that an association exists that needs to be included in a 
model to attempt to explain engagement. The strongest correlations are between engagement and 
school climate, intrinsic motivation and positive affect. Not surprisingly, those students who see 
their school as having a positive and supportive environment, who enjoy school and learning, are 
more engaged, participating more frequently in extracurricular activities. 



Table 2 Correlations with engagement for student attitudinal variables and 
achievement, all students 



Correlation with Engagement 


Perception of class climate 


.14 


Perception of school climate 


.21 


Self-concept of ability 


.16 


QSL - Intrinsic motivation 


.22 


QSL - Opportunity 


.13 


QSL - Achievement 


.15 


QSL - Positive Affect 


.23 


Achievement 


.09 
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Table 3 Correlations with engagement for student attitudinal variables and achievement 
by gender 





Correlation with Engagement 


Males 


Females 


Perception of class climate 


.13 


.12 


Perception of school climate 


.21 


.16 


Self-concept of ability 


.13 


.18 


QSL - Intrinsic motivation 


.20 


.23 


QSL - Opportunity 


.13 


.14 


QSL - Achievement 


.14 


.14 


QSL - Positive Affect 


.21 


.20 


Achievement 


.06 


.09 



Table 3 presents the same correlations, separately for males and females. Among the strongest 
associations with engagement for both males and females are intrinsic motivation and positive 
affect, however the association with school climate is stronger for males than females, and the 
association of self-concept of ability with engagement is greater for females than males. 

Table 4 presents the correlations by school sector. Higher levels of engagement in independent 
schools appears to be associated wdth higher levels of satisfaction on most variables. The 
strongest associations with engagement for each sector were intrinsic motivation and positive 
affect, and for those in independent schools perceptions of school climate were also strongly 
associated with engagement. Most correlations were higher in independent schools. 

Table 4 Correlations with engagement for student attitudinal variables and achievement 
by school sector 



Correlation with Engagement 





Government 


Catholic 


Independent 


Perception of class climate 


.11 


.10 


.18 


Perception of school climate 


.15 


.14 


.20 


Self-concept of ability 


.12 


.15 


.17 


QSL - Intrinsic motivation 


.20 


.19 


.24 


QSL - Opportunity 


.11 


.09 


.17 


QSL - Achievement 


.11 


.14 


.18 


QSL - Positive Affect 


.21 


.16 


.20 


Achievement 


.03 


.09 


.12 



It appears from these analyses that the attitudinal factors most highly correlated with engagement 
are the Quality of School Life scales for Intrinsic motivation and Positive affect. Not surprisingly, 
students who are generally satisfied with school and themselves, and who find school exciting 
and stimulating, are most likely to be those with high levels of engagement, participating more 
fully in extracurricular activities. 

Teacher variables 

The next section of the analysis provides the correlations with student engagement for the teacher 
variables and for the school variables. A multilevel average (similar to aggregation to school- 
level, carried out in MLwdN) was calculated for each of the teacher variables at school-level and 
this school-level average was then assigned to each student within the school. 
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Table 5 Correlations with engagement for teacher variables 



Correlation with Engagement 



Classroom climate 


.07 


School climate 


.07 


Problems with the school 


.07 


Parental involvement 


.08 


Satisfaction resources 


.08 


Satisfaction workload 


.07 


Satisfaction management 


.06 


Satisfaction students 


.07 


Relationships with others 


.08 


School success broadly 


.01 


School success academically 


.08 


Teachers’ rating of other teachers 


.10 



It can be seen from Table 5 that there is not a strong relationship between any of the teacher 
variables and student engagement. This is possibly because these variables have an indirect 
rather than a direct effect on engagement, for example that they affect another factor, which in 
turn affects student engagement. However these relationships and possibilities are not explored 
in this report. The strongest correlation is teachers’ rating of other teachers; that is whether the 
teachers feel that most of the other teachers in their school are experts in their subject matter and 
good communicators with their students. 



School-level variables 

Table 6 shows that the correlations between school-level variables and student engagement are 
moderate for independent schools and for single-sex schools (as this is a dichotomous variable, a 
negative correlation for coeducational schools is the same as a positive correlation for single-sex 
schools). Weaker negative correlations can be seen with school size, implying that engagement 
decreases as school size increases. 



Table 6 Correlations with engagement for school-level variables 



Correlation with Engagement 


School size 


-.07 


Non-government school 


.12 


Coeducational school 


-.12 



Summary 

This section of the report has provided an overview of the student, teacher and school variables 
and their associations with student engagement. There is a number of student demographic and 
background characteristics that are associated with higher and lower levels of engagement. As 
previous research has shown, engagement levels amongst young males are lower than for 
females. Both parental education and occupation are associated with engagement. Occupational 
background, as measured by the ANU3 occupational prestige scale, has a weak correlation with 
engagement. Also related to socioeconomic status, parents’ educational level is strongly 
associated with engagement. Students who have tertiary educated parents have, on average, 
higher levels of engagement than those from families with lower levels of parental education. 
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Of the attitudinal variables, the strongest correlations were found for engagement and 

• perception of school climate, 

• intrinsic motivation, 

• positive affect. 

Only weak correlations were found between student engagement and any of the teacher variables. 
The strongest was found for teachers’ rating of other teachers, indicating that a students’ sense of 
engagement may be related to a mutual sense of confidence among teachers. 

Finally, for the school-level variables, school size was found to have a weak negative correlation 
with engagement, indicating that students at smaller schools show higher levels of engagement. 
There is probably more pressure on students at small schools to participate in a variety of 
activities. Engagement is strongest in independent schools and single-sex schools, which may 
well be confounded with socioeconomic status and with parents’ educational level. 

In the next section of this report, multilevel modelling techniques are used to examine school- 
level differences in student engagement, accounting for backgroimd and other student and school- 
level factors. 



4. INDIVIDUAL AND SCHOOL - LEVEL EFFECTS ON STUDENT ENGAGEMENT 



As discussed previously, modelling the effects of both student-level and school-level variables on 
student engagement necessitates the use of multilevel (or hierarchical linear) modelling 
techniques^. Students are grouped or nested within schools, and standard regression analysis will 
result in smaller standard errors and possibly incorrect conclusions. Analysis of school-level 
variables at the individual student level will result in aggregation bias, underestimating the effects 
of those variables that are estimated at the inappropriate level, while aggregating student-level 
data to the school level fails to fully capture the effects of variables such as achievement that may 
operate at both levels of analysis. For example, achievement measured at the student-level is an 
indicator of a student attribute, whereas school-average achievement at the school-level becomes 
a proxy measure of a school’s normative environment. In other words, the average achievement 
level of a school has an effect on a student above and beyond the effect of their own individual 
achievement. 

MLM techniques allow researchers to model student-level outcomes within schools and then to 
identify and model any between-school differences that occur (Bryk & Raudenbush, 1992). By 
decomposing any observed relationships into separate between-school and within-school 
components, confounding of relationships can be avoided and the correct interpretation of 
empirical relationships can be made. The analyses in this section of the report were carried out 
using MLwiN (Rasbash et al., 2000). 



Engagement 

The outcome variable for this investigation is student-level engagement, and under investigation 
in this chapter is the extent to which this engagement varies between schools. As well, school- 
level engagement is used as a predictor variable in the multilevel modelling, and the use of this 
variable requires careful thought. 

The measure of engagement at the student-level provides an in-context indicator of the student’s 
school experience. At the school-level, engagement becomes a proxy, contextual measure of a 
school’s normative environment. Clearly, the average engagement of students within a given 
school (and its variance) may have an effect on students above and beyond the effect of an 
individual’s engagement. Multilevel analysis resolves the confounding of these two effects by 
facilitating a decomposition of any observed relationship among variables into separate within- 
school and between-school components. Indeed, the inclusion of computed contextual or group 
averages of this kind are commonly undertaken by researchers who have realised that such 
decomposition can be critical to correct interpretations of relationships (e.g. Goldstein, 1987, 
1993; Raudenbush & Willms, 1991, Rowe & Rowe, 1999, Willms, 2000). 

School-level engagement is constructed from student-level engagement scores; it is a school-level 
average of individual student’s scores, calculated as a multilevel average by the MLwiN program. 
Each student within a particular school will have an identical value for this school-level 
engagement, and it is useful to begin identifying schools whose practices somehow act to boost 
levels of engagement for students who might otherwise be disengaged. 



Variance Components model 

The initial model was fitted to estimate the extent to which engagement is related to individual 
schools (Appendix 5 provides technical details of this procedure). Of the total variation in 
engagement, 8.9 per cent was found to be due to between-school differences. This is highly 



^ The support of Dr Ken Rowe is gratefully acknowledged for his guidance in the analysis and reporting 
of the sections of this report on constmct measurement and multi-level modelling. 
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significant, and, with the finding of considerable variation at school- and student-level, indicates 
the necessity of carrying out multilevel analysis. The figure of 9 per cent is consistent with the 
findings of Lee, Chen & Smerdon (1996, p. 21). Figure 6 depicts the results of a residuals'^ 
analysis at the school level, showing the ranked mean point estimates of schools’ engagement 
scores, bounded by 95% confidence (uncertainty) intervals. A key feature of the data shown in 
Figure 6 is that there is greater variability within schools than between schools. It is also clear 
that there is a group of schools with high levels of engagement and a group with low levels of 
engagement; these are significantly different, as their confidence intervals do not overlap. This 
provides a starting point for further research, in which the differences between these two groups 
of schools could be further explored. 




Schools (ranked from lowest to highest on mean student-level 

engagement) 



Figure 6 Residuals of school-level Engagement scores, ranked for 296 schools, showing 
‘error bars’ for 95% confidence intervals 



Four models were used to imderstand the direct and indirect effects of backgroimd and school 
factors on engagement. Each successive model added a further set of explanatory factors to the 
previous model. The first model included the group of student background variables, the second 
model added a set of attitude variables and prior achievement, the third model added a group of 
teacher variables averaged to school-level and the fourth model added two school-level 
contextual variables. Finally School-level Engagement^ as a multilevel average, was included in 
the model to examine the effects of school-level engagement over and above the effects of the 
other factors on student-level engagement. 
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A useful way of interpreting a residual in this context is that it represents what remains after adjusting 
for factors that might influence or explain student’s engagement. 
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Student background and school contextual factors 

The parameter estimates for the first model are shown in the second column of Table 7. Student 
sex, socioeconomic status (as measured by the ANU3 occupational prestige scale), Indigenous 
status, parents’ educational level, home language, parents’ country of birth, student’s aspirations 
were added to the basic equation. In addition several school-level variables were included, such 
as the gender mix of the school and size of school. 

Overall, 8.9 per cent of variance in engagement was accounted for by the variables in this model, 
with 5.6 per cent of residual variance at the school level and the remaining 94.4 per cent at the 
student level. The change in the log-likelihood statistic was significant, indicating that this model 
was a significantly better ‘fit’ than the null model. 

The strongest effect on engagement is student’s gender. The parameter estimate (0.35) divided 
by the standard error (0.03) yields a lvalue of 11.7, which is significant at the p < .001 level. 
Females are a great deal more engaged than males, that is they are much more likely to be 
participating in a wide range of school-related activities. Strong effects were also found for 
independent schools {t = 3.3) compared to government schools and for single-sex schools 
compared to coeducational schools (t= 3.86). 

Socioeconomic status and parents’ educational level were strong predictors of engagement, with 
students from higher socioeconomic levels and with well-educated parents being more likely to 
be engaged. It is likely that parents who are well educated were themselves engaged with 
schooling, and see the benefits of participation in extracurricular activities. 

Student’s aspirations show a significant, strong positive effect, with students who aspire to 
tertiary education showing significantly higher levels of Engagement than those who plan to 
work immediately after finishing school. 

A negative effect was found for indigenous status, indicating lower levels of engagement among 
those students of Aboriginal and Torres Strait Islander background. However home language 
and parents’ country of birth were not found to be significant predictors, indicating that allowing 
for the other variables in the equation, engagement levels are not significantly different for those 
from non-Australian backgrounds. 

As an equity issue, one would hope that the latter would indeed be the case. Engagement in 
schooling and participation in extracurricular activities should not be predicated on being part of 
the cultural majority. Thus the strong negative effect of indigenous status is of some concern, as 
is the strong effect of high socioeconomic status and of independent schools. These warrant close 
examination as more factors are added to the model. 

Student attitudes and achievement 

The third column of Table 7 provides the estimates for Model 2, in which the student attitudinal 
variables and prior achievement in maths and English is added. The variables in this model 
account for 1 5 per cent of the variance in Engagement^ with 6 per cent of the residual variance at 
school level. 

Perceptions about school climate (rating of the school on quality of teaching, effective discipline, 
student learning and school spirit) were found to have a significant effect on engagement. Those 
who rated their school highly were more likely to have higher levels of engagement. Similarly, 
self-concept of ability was found to be a moderately strong predictor, so that those students who 
believed ^emselves academically capable compared to their peers were more likely to be 
involved in a variety of extracurricular activities. 
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Table 7 Parameter estimates and standard errors for the analysis of student engagement 
(students within schools) 


Explanatory variables 


Model 1 


Model 2 


Model 3 


Model 4 


FIXED 










Intercept 


.32 (.13) 


.34 (.14) 


.23 (.16) 


.08 (.14) 


Student-level variables 










Sex 


.35 (.03) 


.33 (.03) 


.34 (.04) 


.27 (.03) 


Socioeconomic status 


.06 (.02) 


.05 (.02) 


.05 (.02) 


.03 (.02) 


Indigenous status 


-.24 (.11) 


-.31 (.12) 


-.25 (.13) 


-.22 (.13) 


Parents’ educational level 


.08 (.02) 


.06 (.02) 


.07 (.02) 


.06 (.02) 


Home language 


.02 (.07) 


-.03 (.07) 


00 

o 

00 

o 

I* 


-.11 (.08) 


Parents’ country of birth 


-.02 (.02) 


-.03 (.02) 


.01 (.03) 


-.01 (.03) 


Student’s aspirations 


.11 (.02) 


.00 (.02) 


.01 (.03) 


.00 (.03) 


Class climate 




.03 (.02) 


.04 (.02) 


.03 (.02) 


School climate 




.10 (.02) 


.09 (.02) 


.07 (.02) 


Self-concept of ability 




.14 (.02) 


.15 (.03) 


.13 (.02) 


QSL Intrinsic motivation 




.08 (.03) 


.08 (.03) 


.09 (.03) 


QSL Opportunity 




-.02 (.02) 


-.02 (.02) 


-.02 (.02) 


QSL Achievement 




.02 (.02) 


-.02 (.03) 


.01 (.02) 


QSL Positive Affect 




.06 (.03) 


.06 (.03) 


.04 (.03) 


Achievement 




-.02 (.02) 


-.03 (.02) 


-.04 (.02) 


School-level variables 










Catholic school 


-.08 (.07) 


-.10 (.07) 


-.09 (.08) 


-.09 (.06) 


Independent school 


.23 (.07) 


.20 (.07) 


.29 (.09) 


-.02 (.07) 


Coeducational school 


-.27 (.07) 


-.20 (.07) 


-.18 (.08) 


.04 (.06) 


School size 


-.09 (.02) 


-.08 (.02) 


-.11 (.03) 


-.04 (.02) 


Classroom climate 






.04 (.03) 


.00 (.02) 


School climate 






-.01 (.07) 


-.03 (.05) 


Problems with school 






.02 (.09) 


.01 (.06) 


Parental involvement 






.02 (.06) 


.04 (.04) 


Satisfaction resources 






.02 (.04) 


.01 (.03) 


Satisfaction workload 






-.03 (.04) 


-.01 (.03) 


Satisfaction management 






-.04 (.07) 


-.01 (.04) 


Satisfaction students 






.04 (.10) 


-.05 (.07) 


Relationships with others 






-.04 (.09) 


.01 (.06) 


School success broadly 






.09 (.04) 


.02 (.03) 


School success academically 






.02 (.08) 


.02 (.05) 


Teachers’ rating of other teachers 






.04 (.06) 


.01 (.04) 


School-level Engagement 








.31 (.02) 


CJ^ (between schools) 


.05 (.01) 


.05 (.01) 


.05 (.01) 


.00 (.00) 


(j^ (between students) 


.84 (.02) 


.78 (.02) 


.77 (.02) 


.76 (.02) 


-2 log-likelihoocf 


11131.30 


8793.85 


7455.57 


7298.14 


E 


<.001 


<.001 


<.001 


<.001 



The significance in the difference in the -2 log-likelihood is tested using a Chi-squared statistic, where 
the degrees of freedom are based on the number of variables added to the equation. The significance 
(or p-value) is noted in the table although the Chi-square statistics are not. 
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The two Quality of School Life scales for intrinsic motivation and positive affect also had positive 
effects on engagement. These variables can be seen to have strong independent effects on 
student’s engagement, however they also transmitted or relayed the effects of the different 
student background and contextual variables. This is evident from the drop in the sizes of the 
parameter estimates for gender, student’s aspirations, independent schools and coeducational 
schools in particular. This tells us that not only do gender, attendance at an independent school 
and attendance at a single-sex school have independent effects on engagement, they also have a 
transmitted effect through their influence on perceptions of school climate, self-concept of ability, 
intrinsic motivation and positive affect. 

School level factors 

The addition of the next group of variables - teacher attitudes - had little impact, as can be seen in 
the fourth column of Table 7. The variables in this model account for 16.4% of the variance in 
engagement, with 6 per cent of the residual variance at school level. Only one of the teacher 
attitudes had a significant impact on student engagement. 

Teacher’s belief that their school was broadly successful was found to be a significant predictor 
of student engagement. This variable relates to teachers’ assessments on the success of their 
school in such things as providing a curriculum that caters for all students. It would seem that a 
strong feeling among teachers that the school is inclusive and successful is a positive influence on 
student engagement. 

The fifth column of Table 7 presents the final model in this data analysis; that in which school- 
level engagement was added into the equation. The variables in Model 4 account for 21.8 per 
cent of the variance in Engagement, with all of the residual variance at student level. It should be 
remembered that a school’s average level of Engagement acts as a proxy level-2 contextual 
variable, that represents a school’s normative environment. It can be see that the effect of this 
variable was strong and significant. While this is to be expected, given the manner in which the 
variable was constructed, what is important to note is the effect of this variable on others. High 
engagement at the school level appears to moderate the negative effects of socioeconomic status 
and indigenous status, and the magnitude of both effects declines to the extent that neither is 
statistically significant (ie the t value calculated as the ratio of parameter estimate to standard 
error is not significant). This finding suggests that schools with high average levels of student 
engagement increase the likelihood of extracurricular participation of indigenous students and 
those students from low socioeconomic status. This is a particularly important finding for 
educational policy. School policies on participation in extracurricular activities, indeed school 
policies on the provision of such activities, are policies that are readily amenable to change by the 
school community itself. 

Students’ perceptions of school climate, self-concept of ability and level of intrinsic motivation 
all remain as significant predictors of engagement, as does gender and parents’ educational 
levels. The effects of independent schools on engagement are completely removed, as are the 
effects for single-sex schools. 



Other influences on school-level engagement 

School-level engagement appears to be an important influence on the engagement levels of the 
students within the schools. This variable represents, as best we can measure it through these 
data, a school ‘ethos’ of participation. There are a number of other factors at the school level that 
may also have some influence on engagement, such as whether the school is single-sex or 
coeducational, the size of the school, whether the school is a government, independent or 
Catholic school, and the mean socioeconomic level of students in the school. 
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To estimate the effects of these variables on school-level engagement, a multilevel model was 
fitted to the data vrith school-level engagement as the outcome variable (Table 8). Almost 9 per 
cent of the between-schools variation was able to be explained by this model. The parameter 
estimates shown in Table 8 suggest that the two factors that have the greatest influence on overall 
school levels of engagement were the gender mix of the school and the size of the school. 
Single-sex schools show higher levels of engagement than coeducational schools, and smaller 
schools than larger schools. Also significant but with a much smaller effect, was average school 
socioeconomic status. This finding indicates that schools with higher socioeconomic status are 
more likely to have higher levels of engagement than those schools with lower average 
socioeconomic status. 



These findings are intuitively correct; schools with more resources are better able to provide such 
activities to their students, and the parents of higher socioeconomic level students are more likely 
to be able to provide the financial resources necessary for their children to participate in a broad 
range of extracurricular activities. The positive finding for single-sex schools is worthy of further 
investigation. Only 3 per cent of government schools, but 63 per cent of Catholic schools and 40 
per cent of non-Catholic independent schools in the LSAY sample are single-sex. 



Table 8 Parameter estimates and standard errors for the analysis of school-level 
engagement 



Explanatory variables 


Model 1 


Intercept 


1.49 (.05) 


School-level variables 




Average socioeconomic level (ANU3) 


.05 (.02) 


Size of school 


-.07 (.02) 


Catholic school 


-.08 (.05) 


Independent school 


.11 (.06) 


Coeducational school 


-.23 (.05) 



Are influences on Engagement the same for males and females? 

The effect of gender is the strongest of all predictors, other than school-level engagement. 
Females are much more highly engaged than males whatever other factors are included in the 
equation. It may be that there are different influences for males and females, however the effects 
for females are ‘‘swamping” those for males. Separate multilevel analyses were therefore carried 
out for males and females, using exactly the same indicators as for the whole group. The 
parameter estimates and their standard errors separately for males and females for each of the 
four models are presented in Table 9 and Table 10. The initial variance components models 
found that for both males and females, approximately 1 1 per cent of the residual variance was at 
school-level, again indicating that multilevel analysis is essential to account for the clustering 
effects of students within schools. 

The second columns of Tables 9 and 10 show that, as expected, there are significant differences 
in the effects of the various contextual and background variables on engagement for males and 
females in the first model. The amount of residual variance explained by Model 1 was lowest for 
males, accounting for just over 3 per cent, while for females the model accounted for just over 7 
per cent. For males, a little over 10 per cent of variance was at the school-level, while for 
females it was a little over 7 per cent. 
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For males, the strongest effect was for coeducational schools, indicating that males at 
coeducational schools had significantly higher levels of engagement than those males at single- 
sex schools. For females, this effect was not significant, however there was a suggestion that 
females at single-sex schools may have higher levels of engagement. 

For both males and females, socioeconomic status had a significant positive effect on 
engagement, with the effect being stronger for females than for males. For both males and 
females, those from higher socioeconomic status families showed higher levels of engagement. 
As might be expected, parents’ educational level also had a similar effect, with those students 
from families with higher levels of parental education having significantly higher levels of 
engagement. 

School size was found to have a negative effect on engagement for females, but no effect for 
males. This indicates that females at smaller schools are more likely to have higher levels of 
engagement than those females at larger schools, however for males there was no difference 
according to the size of the school. 

Student’s aspirations were found to be a significant positive predictor of engagement for both 
males and females, with those who aspire to tertiary study having higher levels of engagement. 

Non-significant predictors for either males or females were Indigenous status, home language, 
parents’ country of birth, and attendance at a Catholic school. This tells us that, accounting for 
the effects of socioeconomic status, parents’ educational level, and school sector or school type, 
there are no differences in engagement for indigenous students, for those for whom English is a 
second language or whose parents were bom in non-English speaking countries, or for those at 
Catholic schools compared to government schools. 

The third and fourth columns of Tables 9 and 10 show the developments in the models with each 
successive addition of groups of factors. These will not be examined in detail, however are 
provided for completeness. The addition of the student attitudinal variables and achievement 
remove the effect of socioeconomic status, student’s aspirations and coeducational school that are 
seen in the initial model for males, and the effects of parents’ educational level and student’s 
aspirations for females. 

The last column in each table provides the parameter estimates for whole-school engagement, and 
show that it has a very large and significant effect on engagement for both males and females, 
with the effect being larger for females. 

Throughout the model, parents’ educational level was found to be a significant effect for males, 
as was perceived positive school and classroom climate. These last two variables are of 
particular interest as they are amenable to change by schools. School climate is measured by 
perceptions of the quality of teachers, effective discipline, student learning and school spirit. 
Class climate is measured by perceptions of whether students in the class were eager to learn, 
made good progress, worked hard and were well-behaved. This finding means that for males, a 
structured and supportive school and classroom environment is particularly helpful to increase 
levels of engagement. 

For females, socioeconomic status was found to be a positive predictor throughout the analysis. 
Of the attitudinal variables, self-concept of ability and school climate were the only significant 
predictors in the final model. For females, the factors most likely to improve engagement were a 
strong belief in their own ability in comparison to that of their classmates, and a belief that the 
school climate is positive and structured. Fostering both these types of beliefs in females 
increases their levels of engagement with school. 
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Table 9 Parameter estimates and standard errors for the analysis of student engagement 



(students within schools) for males 


Explanatory variables 


Model 1 


Model 2 


Model 3 


Model 4 


FIXED 

Intercept 

Student-level variables 


-.59 (.19) 


-.37 (.21) 


-.25 (.23) 


-.11 (.21) 


Socioeconomic status 


.05 (.02) 


.03 (.03) 


.02 (.03) 


.01 (.03) 


Indigenous status 


-.15 (.16) 


-.19 (.18) 


-.26 (.18) 


-.27 (.18) 


Parents’ educational level 


.05 (.02) 


.07 (.02) 


.07 (.02) 


.07 (.02) 


Home language 


.08 (.10) 


.01 (.11) 


-.04 (.11) 


-.05 (.11) 


Parents’ country of birth 


.00 (.03) 


.03 (.04) 


.03 (.04) 


.02 (.04) 


Student’s aspirations 


.07 (.02) 


.02 (.03) 


.02 (.03) 


.02 (.03) 


Class climate 




.06 (.03) 


.07 (.03) 


.07 (.03) 


School climate 




.16 (.03) 


.16 (.03) 


.12 (.03) 


Self-concept of ability 




.06 (.04) 


.03 (.04) 


.03 (.04) 


QSL Intrinsic motivation 




.05 (.04) 


.07 (.04) 


.07 (.04) 


QSL Opportunity 




-.03 (.03) 


-.03 (.03) 


.02 (.03) 


QSL Achievement 




.03 (.03) 


.04 (.04) 


.05 (.04) 


QSL Positive Affect 




.07 (.04) 


.07 (.04) 


.04 (.04) 


Achievement 




-.01 (.03) 


-.01 (.03) 


-.05 (.03) 


School - level variables 
Catholic school 


.13 (.11) 


.08 (.11) 


.11 (.13) 


.08 (.11) 


Independent school 


.10 (.10) 


.00 (.10) 


-.01 (.15) 


.01 (.12) 


Coeducational school 


.20 (.09) 


.15 (.09) 


.12 (.10) 


-.02 (.08) 


School size 


.02 (.03) 


.02 (.03) 


.03 (.03) 


.02 (.03) 


Classroom climate 






.07 (.03) 


.03 (.03) 


School climate 






-.37 (.13) 


-.17 (.11) 


Problems 






-.30 (.13) 


-.05 (.11) 


Parents 






.15 (.07) 


.09 (.07) 


Satisfaction resources 






-.11 (.08) 


-.06 (.07) 


Satisfaction workload 






-.07 (.10) 


-.11 (.08) 


Satisfaction management 






-.01 (.08) 


-.01 (.06) 


Satisfaction students 






.27 (.13) 


.09 (.10) 


Relationships 






-.09 (.13) 


.00 (.10) 


School success broadly 






-.06 (.06) 


.02 (.05) 


School success academically 






.18 (.12) 


.02 (.10) 


Teachers’ rating of other teachers 






.24 (.07) 


.11 (.06) 


School-level Engagement 








.29 (.03) 


O’J (between schools) 


.09 (.02) 


.06 (.02) 


.04 (.02) 


.00 (.00) 


(between students) 


.82 (.03) 


.75 (.03) 


.74 (.03) 


.73 (.03) 


-2 log-likelihood 


4791.48 


3701.21 


3243.07 


3166.36 


E 


<.001 


<.001 


<.001 


<.001 



BEST COPY AVAILABLE 
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Table 10 Parameter estimates and standard errors for the analysis of student engagement 
(students within schools) for females 

Explanatory variables Model 1 Model 2 Model 3 Model 4 



.23 (.20) .61 (.21) 

.10 (.02) .08 (.02) 

-.27 (.16) -.32 (.17) 



FIXED 

Intercept 

Student-level variables 
Socioeconomic status 
Indigenous status 
Parents’ educational level 
Home language 
Parents’ country of birth 
Student’s aspirations 

Class climate 
School climate 
Self-concept of ability 
QSL Intrinsic motivation 
QSL Opportunity 
QSL Achievement 
QSL Positive Affect 
Achievement 
School-level variables 
Catholic school 
Independent school 
Coeducational school 
School size 

Classroom climate 
School climate 
Problems 
Parents 

Satisfaction resources 
Satisfaction workload 
Satisfaction management 
Satisfaction students 
Relationships 
School success broadly 
School success academically 
Teachers’ rating of other teachers 

School-level Engagement 
2 

<7^ (between schools) 

Gg (between students) 

-2 / og-l ikel ihood 

R 



.04 (.01) .02 (.01) 

-.08 (.10) -.08 (.10) 

-.02 (.03) -.05 (.03) 

.08 (.02) -.01 (.03) 

.01 (.03) 
.08 (.03) 
.16 (.03) 
.08 (.04) 
-.03 (.03) 
-.02 (.03) 
.08 (.03) 
-.02 (.03) 

-.09 (.09) -.11 (.09) 

.23 (.09) .26 (.09) 

-.15 (.10) -.12 (.09) 

-.10 (.03) -.09 (.03) 



.06 (.01) 


.04 (.01) 


.82 (.03) 


.76 (.03) 


5924.36 


4647.95 


<.001 


<.001 



.36 (.23) 


.09 (.21) 


.08 (.03) 


.06 (.02) 


-.16 (.18) 


-.11 (.18) 


.02 (.02) 


.01 (.01) 


-.10 (.11) 


-.15 (.11) 


-.03 (.04) 


-.03 (.04) 


.01 (.03) 


.00 (.03) 


.02 (.03) 


.01 (.03) 


.06 (.03) 


.06 (.03) 


.17 (.03) 


.15 (.03) 


.07 (.04) 


.07 (.04) 


-.03 (.03) 


-.02 (.03) 


-.03 (.03) 


-.02 (.03) 


.08 (.04) 


.06 (.04) 


-.03 (.03) 


-.04 (.03) 


-.04 (.10) 


-.04 (.08) 


.38 (.11) 


.05 (.09) 


-.11 (.10) 


.20 (.07) 


-.12 (.03) 


-.04 (.03) 


.01 (.03) 


-.01 (.03) 


.02 (.08) 


-.01 (.07) 


.04 (.11) 


.03 (.08) 


.11 (.07) 


.05 (.06) 


.04 (.04) 


.02 (.03) 


.04 (.05) 


-.02 (.04) 


-.08 (.08) 


-.02 (.06) 


.17 (.12) 


.14 (.09) 


.04 (.11) 


.04 (.08) 


.05 (.05) 


.00 (.04) 


.11 (.10) 


.07 (.08) 


.02 (.07) 


.01 (.05) 




.34 (.03) 


.04 (.01) 


.00 (.00) 


.75 (.03) 


.72 (.03) 


4037.25 


3917.49 


<.001 


<.001 



o 
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In the final model, a very strong and significant effect was found for females attending 
coeducational schools. This effect was not found for males. Females in coeducational schools, 
taking into account other factors in the model, showed higher levels of engagement. It is 
important to recognise that this effect is only observed after allowing for school-level 
engagement. This means that in schools of the same average level of engagement there is a 
benefit for girls in coeducational environments, however for males there is no difference between 
coeducational and single-sex environments, given schools of the same average level of 
engagement. 



Summary 

This chapter has explored a number of multilevel models to explain school-level variance in 
student engagement. While the amount of between-school variation is modest, it is sufficient to 
warrant multilevel modelling techniques to account for the hierarchical structure of the data: 
students within schools. The following provides a summary of the key findings for this chapter. 

• School-level engagement was found to be the most important factor in individual students’ 
levels of engagement. The final model used throughout this chapter found this variable to be 
the strongest predictor of engagement. 

• Gender was found to be the next strongest predictor of engagement. Females were more 
engaged than males whatever other variables were entered into the model. 

• The following factors remained significant predictors of engagement for all students as a 
group, controlling for all other factors in the model: 

- Parents’ educational level; 

- School climate; 

- Self-concept of ability; and 

- Intrinsic motivation. 

• School size was a significant negative predictor of individual level engagement. That is, 
students in smaller schools tend to have higher levels of engagement than students in large 
schools. 

• Positive effects were found for independent schools and single-sex schools prior to the 
addition of the whole-school engagement variable. Given whole-school engagement, 
however, there were no differences found for the whole sample for either school sector or for 
school type. Whole school engagement appears to have the effect of removing these 
differences. 

A separate multilevel analysis was conducted to examine the predictors of school-level 
engagement. 

• The strongest factors in influencing school-level engagement were found to be gender mix of 
the school, size of the school and average socioeconomic status of the school. The data 
indicates higher levels of engagement in single-sex schools as compared to coeducational 
schools, smaller schools and in schools with high socioeconomic levels. 

Due to the size of the effect for gender, the same techniques were applied to the data separately 
for males and females. With the effects for females being so strong, it was thought that they 
might act to ‘swamp’ the effects for males in the combined analysis. The significant findings 
fi-om these analyses were: 

• For both males and females, but more strongly for females, whole-school engagement was 
the most significant predictor of individual engagement. 




39 



30 Longitudinal Surveys of Australian Youth Research Report No 27 



• The significant predictors along with whole-school engagement were found to be: 
for males 

- Parents’ education level 

- Classroom climate 

- School climate 
and for females: 

- Socioeconomic status 

- Self-concept of ability 

- School climate 

- Coeducational school. 

That there are differences between the significant predictors for males and females is an 
important finding. It has implications for school policies if schools wish to increase the levels of 
engagement amongst males and females in their schools. The final chapter explores these 
findings and their implications in a broader context. 
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5. DISCUSSION AND POLICY IMPLICATIONS 



What are the important outcomes of schooling? For more than one-quarter of the students in 
school today, and for a greater proportion of some social groups, it is not a tertiary entrance rank; 
these students will leave school before such issues become a reality for them. For instance 
around 10 per cent of students leave before completing Year 10 at school, and of those young 
people, a disproportionate number are low achievers, boys, indigenous (Marks & Fleming, 1999), 
or from low socioeconomic status families (Lamb et al., 2000). On a practical level, young 
people who do not complete secondary school are more likely to experience periods of 
unemployment, to find jobs in a narrow field of occupations and to be more reliant on 
government income support, compared to their peers who do complete Year 12. 

Many young people leave school early for reasons other than to find work; a significant minority 
leave because they fail to see the relevance of school to them. Many have, over a period of time, 
become alienated from education and from the schools in which they are enrolled. While this 
disengagement or alienation might be seen in terms of dropping out or problematic behaviour at 
school, it might also be seen in terms of failure to achieve potential. Some students simply 
endure thirteen years of schooling at min imum participation levels. If we are to encourage 
lifelong learning skills in students then we need to address low engagement with school. 

This report draws on Finn’s (1989) taxonomy of engagement or participatory behaviours. The 
third level in this taxonomy is the student’s participation in extracurricular activities, which may 
be the primary source of attachment to school for those whose academic performance is weak. 
Finn argued that the likelihood of a student completing secondary schooling is maximised if the 
student “maintains multiple, expanding forms of participation in school-relevant activities”(p. 
117). 

This report has examined individual and school effects on engagement. Four key questions were 
examined: 

• What is the influence of socioeconomic background on student’s engagement levels? 

• What is the influence of background factors such as gender on student’s engagement 
levels? 

• To what extent can student engagement be attributed to the differences between schools? 

• What are the factors that affect a student’s engagement with school? 

Firstly, it was found that socioeconomic background did indeed affect students’ engagement 
levels. Measured by the ANUS occupational prestige measure, the correlation with engagement 
was weak but positive. However there is a strong possibility that the effect of socioeconomic 
status is multiple, in that it also affects the school that a student attends and therefore the 
environment that a student is exposed to. In the context of this study, it also affects the financial 
resources that a family has available to allow young people to participate in a wide range of 
activities at school, or to attend a particular school which has a strong focus on students’ 
participation. 

Gender was found to be a strong influence on student’s engagement, with females showing 
significantly higher levels of engagement than males; in all school sectors, in coeducational as 
well as single-sex schools, and at all achievement levels. 

Several of the affective variables from the Quality of School Life scales exerted an effect on 
engagement, and weak correlations were also found between a number of the teacher variables 
and school variables and engagement. 
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To examine the second and third questions, multilevel modelling techniques were used. Almost 9 
per cent of the variation in student engagement was found to be due to between-school 
differences. Whilst this is not large, it is highly significant, and indicates that it does matter what 
school a child goes to. A number of models were then fitted to these data to try and identify the 
particular variables that affect students’ engagement, and to identify the variables at school level 
that affect school-level engagement. The findings indicated that gender was a strong predictor of 
engagement. Whatever other variables were entered into the model, gender remained one of the 
strongest influences. 

School-level engagement, however, was found to be the strongest influence on engagement. This 
variable represents, as best we are able to measure it with these data, a school ‘ethos’ of 
participation. It is an average for a school, and as such, there are deviations from the mean that 
would provide a starting point for further qualitative research studies. The level of engagement 
within a school was found to vary significantly between schools, and was found to be higher in 
single-sex schools, small schools, and in schools with higher levels of socioeconomic status. 

Allowing for whole-school engagement, a number of other variables were found to remain as 
significant influences on individual-level engagement. Gender, parents’ educational level, 
perceptions of school climate, self-concept of ability, and intrinsic motivation were all found to 
have a significant effect on individual engagement over and above that of whole-school 
engagement. The effect of gender was found to be particularly strong, and so the analysis was 
repeated separately for males and females. 

Different predictors were found when the analysis was conducted separately for males and 
females, indicating that for males, the strongest predictors of engagement, apart from overall high 
school-levels of engagement, were parents’ educational level and perceptions of classroom and 
school climate. For females, the next strongest predictors were socioeconomic status, self-concept 
of ability, perceptions of school climate, and attendance at a coeducational school. 

These are important findings from a policy perspective. Often, educational research finds that the 
factors that have the greatest influence on student outcomes are those that are not readily 
amenable to change, such as socioeconomic status or prior achievement. In this study, however, 
the focus on extracurricular activities provides an avenue that is amenable to change. 

Firstly, the promotion of participation in extracurricular activities is one that is easily influenced 
by school administrations. Creating an environment at school where there is a wide variety of 
extracurricular activities provided, in which students are expected to participate in a variety of 
such activities, and are encouraged strongly in this participation, vrill lift personal levels of 
engagement with school. Further research into the practices of schools in the independent sector 
could be of benefit to understanding the influence of school-level policies on participation in 
extracurricular activities. 

Marsh (1992) has shown us that such high levels of engagement “have positive benefits across a 
wide variety of educationally relevant outcomes for students across a vride variety of 
educationally relevant backgrounds” (p. 559). In particular as a long-term benefit, young people 
who participate in a variety of extracurricular activities are those who are more likely to be 
involved in voluntary social and community activities as young adults (Lindsay, 1984). Finn 
(1989) argued that schools need to actively promote extracurricular participation, and recognise it 
as a means of promoting high levels of student attachment to school. He points to deleterious 
school policies such as: 

Policies that exclude the youngster from extracurricular participation, detentions that 
don’t involve school-related work, and suspensions, make it more difficult for the 
individual to maintain regular contact with the school environment. For a student in 
this situation, dropping out may seem to be a very small step (Finn, 1989, p. 131) 
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Secondly, the findings are important because they isolate several differences in factors affecting 
the engagement of males and females that can be manipulated by schools. For males, attention in 
schools needs to be paid to classroom and school climate. Males appear to need more of a 
supportive school and classroom environment to be engaged with their school. They need to be 
strongly encouraged by their schools and by their parents to participate in extracurricular 
activities, and a broader range of activities developed by schools that are appealing to young 
males. 

For females, schools need to focus on developing a strong self-concept of ability and positive 
views of school climate. Whilst for males, parents’ educational level, and for females, 
socioeconomic status, are not malleable, their effects are small compared to the effects of overall 
high levels of school engagement. 

This report has indicated that it does matter which school a student attends. Students at schools 
which have the resources or the commitment to provide a broad range of extracurricular activities 
and encourage students to participate, generally have higher levels of engagement than those in 
schools which do not. Strong participation in such activities leads to a student’s closer 
connectedness to the school community, and it is argued that there are ‘flow-on’ effects to more 
academic parts of the curriculum. As an equity issue, it is important that this be addressed. 
Socioeconomic status is a persistent influence on participation, both at the individual level and at 
the school level. At present, students with parents who have the financial resources to allow a 
wide participation in extracurricular activities obtain a benefit fi*om schooling that those students 
with less access to financial resources do not. 
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Appendix 1: Samples and populations 



The Sample 

The sample is a national representative sample of Year 9 students in 1998. It is a stratified 
random sample. The major stratum in the design was State (or jurisdiction) of schooling. 
Students from smaller States were to be over-sampled and, correspondingly, students from larger 
States were under-sampled. Selection of students within States was proportional by school sector. 
Three sectors were used as strata: government schools. Catholic schools and non-government, 
non-Catholic (referred to as independent) schools. The population data for strata were taken from 
the Schools Australia series (ABS). Within strata, schools were selected proportional to their size. 
Information on the number of Year 9 students in each school came from ACER’S Sampling 
Frame which, in turn, was based on information provided by the relevant State authorities and, in 
the case of non-govemment schools, by the Federal Department of Educational, Training and 
Youth Affairs (DETYA). These figures were from the 1997 annual school census. Within schools 
two classes were randomly selected (again, proportional to their size). Schools were asked for a 
list of the number of students enrolled in each of their Year 9 classes for a subject studied by all 
Year 9 students in the school (usually English classes). 

The 1998 data was collected from self-completion questionnaires that the students were asked to 
fill out at the time they undertook the achievement tests in literacy and numeracy. The 1999 data 
was collected with mailed questionnaires. This form collects basic information on participants’ 
educational and labour force activities over the previous 12 months and contact details for 
subsequent telephone-based interviews. In the second year after the initial survey contact (by 
which time sample members are normally in Year 11) the method of data collection changes to 
Computer-Assisted Telephone Interviewing (CATI). 

Cases were weighted to correct for the disproportionate sampling between strata and to correct 
for the variation between strata due to differential response rates and variable class sizes. 
Additional weights were constructed to compensate for sample attrition. Further details on the 
general weighting procedures can be found in LSAY Technical Paper Number 15 (Marks 8l 
Long, 2000). 
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Appendix 2: Description of individual-level variables 



Socioeconomic status 


Responses to questions about parents’ occupations were assigned 
occupational status scores based upon the AIW3 scale. The ANXJ3 
scale ranges from 0 (low status) to 100 (high status). This scale assigns 
status scores based on average income and educational levels, to some 
300 occupational groups. Examples of jobs at the top of the status 
hierarchy are medical practitioners, university teachers and legal 
professionals. Examples of jobs at the bottom of the status hierarchy are 
various mining, construction and related labourers, forklift drivers, 
cleaners and product assemblers (Jones, 1989). 

To make the best use of available information, the occupation of the 
male parent/guardian was taken as the basis for the occupational 
measure. Where information was missing on the male parent the 
occupation of the female parent/guardian was substituted. This 
approach was taken because a large proportion of respondents indicated 
that the occupation of the female parent was "home duties’, an 
occupation for which there is no occupational prestige score. 


Parent's educational 
level 


The measure of parental education was based on student’s self-reports 
of the highest level of education that both parents had attained. 
Mother’s education was taken as the base measure, which, if missing, 
was replaced by father’s education. 


Home language 


Home language was based on student’s self-reports of the primary 
language spoken at home. In these analyses it was treated as a simple 
dichotomy in which students whose primary home language was 
English were contrasted \vith students for whom it was not. 


Parents ' country of 
birth 


This measure was based on father’s country of birth. A respondent was 
defined as Australian if their father was bom in Australia, Other 
English speaking country if their father was bom outside Australia in a 
predominantly English-speaking country, and Non-English if their 
father was bom in a predominantly non-English speaking country. 


Student's aspirations 


Students’ educational aspirations were measured in Year 10 by the 
student’s responses to the question "" what do you plan on doing after 
leaving school?” Four categories were used, representing those who 
had no post-school study plans, those who planned to do an 
apprenticeship, those who planned on TAFE or other study options, and 
those who planned to attend university. 


School sector 


This measure refers to school system attended in Year 9. Three 
categories are used- government schools. Catholic schools and 
Independent non-Catholic schools. Dummy variables were created for 
Catholic schools (coded as 1 with government schools coded as 0) and 
for independent schools (coded as 1 with government schools coded as 
0). 


Gender mix of school 


This measure refers to whether the student attended a single-sex or 
coeducational school in Year 10. No distinction is made in these 
analyses between single-sex male and female schools. A dichotomous 
variable was created for gender-mix (with coeducational schools coded 
as 1 and single-sex schools coded as 0) 


Achievement 


Students were asked to complete tests on literacy and numeracy when 
they were first contacted in 1998. From their answers in these two tests 
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three measures were constructed: achievement in literacy in Year 9; 
achievement in numeracy in Year 9; and combined achievement in 
literacy and numeracy in Year 9. 

The combined measure of achievement in literacy and numeracy 
represents an overall measure of early school achievement. The scores 
for the literacy and numeracy tests were centred about the means and 
summed to produce a combined measure of achievement. For the 
presentation of means and boxplots the continuous measure was split 
into four categories, based upon quartiles of achievement (that is, the 
highest quartile represents the top 25 per cent of students, then next 
quartile represents the next 25 per cent of students, and so forth). 
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Appendix 3: Factor score regression weights 



Table A1 Factor score regression weights for all constructed variables 



Scale 


N* 










Item weights 








Student-level 




enl 


en2 


en3 


en4 


en5 










Engagement 


8452 


.098 


.286 


.339 


.298 


.193 














cll 


cl2 


cl3 


cl4 












Classroom climate 


8452 


.239 


.301 


.370 


.181 
















scl 


sc2 


sc3 


sc4 












School climate 


8860 


.249 


.242 


.399 


.218 
















sconl 


scon2 


scon3 














Self-concept of ability 


10743 


-.045 


-.037 


1.072 














Quality of School Life 




iml 


im2 


im3 


im4 


im5 


im6 








- Intrinsic motivation 


10174 


.221 


.078 


.347 


.088 


.269 


.186 












opl 


op2 


op3 


op4 


op5 


op6 








- Opportunity 


10258 


.131 


.228 


.188 


.307 


.070 


.214 












achl 


ach2 


ach3 


ach4 


ach5 


ach6 








- Achievement 


10258 


.104 


.240 


.246 


.161 


.170 


.262 












pal 


pa2 


pa3 


pa4 


pa5 


pa6 








- Positive affect 


10124 


.154 


.181 


.297 


.214 


.216 


.071 








School-level 




cell 


ccl2 


ccl3 


ccl4 












Classroom climate 


1698 


.307 


.320 


.339 


.078 
















sccll 


sccl2 


sccl3 


sccl4 












School climate 


1678 


.264 


.337 


.353 


.090 
















prol 


pro2 


pro3 














Problems 


1677 


.174 


.102 


.031 


















pari 


par2 


par3 


par4 












Parents 


1699 


.153 


.185 


.342 


.419 
















srl 


sr2 


sr3 


sr4 












Satisfaction- resources 


1686 


.070 


.121 


.796 


.042 
















swl 


sw2 


sw3 


sw4 












Satisfaction -workload 


1661 


.284 


.549 


.161 


.092 
















sml 


sm2 


sm3 














Satisfaction- management 


1696 


.228 


.320 


.473 


















ssl 


ss2 


ss3 


ss4 












Satisfaction-students 


1698 


.160 


.308 


.307 


.318 
















rell 


rel2 


rel3 














Relationships with others 


1711 


.278 


.348 


.433 


















bssl 


bss2 


bss3 














Broad school success 


1695 


.561 


.341 


.143 


















sasl 


sas2 


sas3 


sas4 


sas5 


sas6 


sas7 


sas8 


sas9 


School academic success 


1659 


.103 


.109 


.072 


.208 


.114 


.075 


.157 


.164 


.131 






ratel 


rate2 


rate3 


rate4 


rate5 


rate6 








Rating of other teachers 


1695 


.072 


.224 


.198 


.119 


.173 


.301 









* N= the number of cases with complete data 



er|c best copy available 




A /Tv. 
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Appendix 4: Constructed student and school-level measures 



Student-level 

Perceived class climate 



Perceived school climate 



Self-concept of ability 



Quality of School Life 



This was measured in the Year 10 questionnaire by asking students 
to respond to four items which asked ^^How would you describe 
most of your classes ? '' 

• Students are eager to learn, 

• Students are make good progress, 

• Students work hard and 

• Students are well behaved. 

Students’ perceptions of their school climate were measured in a 
similar manner in the Year 10 questionnaire. Students were asked 
would you rate your school on each of the following? ” 

• Quality of teachers, 

• Effective discipline, 

• Student learning and 

• School spirit. 

A measure relevant to the continuation of school past the 
compulsory level is a student’s self-concept of ability. The variable 
representing this self-concept of ability is a combination of the 
student’s self-report of their ability in relation to their classmates in 

• English, 

• mathematics and 

• overall. 

The ACER Quality of School Life Questionnaire has been used in a 
number of studies in different contexts over several decades. The 
form used in the LSAY 1998 survey and used in these analyses 
asked students about their life at school. They were asked to 
respond to My school is a place where ... for the following scales 

Positive affect (General well-being) 

• I feel happy 

• I like learning 

• I enjoy being there 

• I like to go 

• Learning is fun 

• I feel safe and secure. 

Intrinsic motivation (self-motivation in learning) 

• The work we do is interesting 

• I like to ask questions 

• I enjoy what I do 

• I like to do my best 

• I get excited about the work we do 

• I like to do extra work 

Opportunity (student’s belief in the relevance of learning) 

• We learn important things 

• The work is good preparation for the future 

• We learn useful skills 

• We get a chance to do interesting work 



Student Engagement with School 4 1 



SchooNevel 
Classroom climate 



School climate 



School problems 



Parental involvement 



• The things we learn will help in adult life 

• The things I am taught are worthwhile learning 

Achievement (student’s sense of success in school work) 

• I have learnt to work hard 

• I achieve a satisfactory standard in my work 

• I always achieve a satisfactory standard in my work 

• I know how to cope with the work 

• I know I can be successful 

• I am a success as a student 



A measure which examines teachers’ perceptions of the climate in 
their Year 10 classes. It consists of responses to In your Year 10 
class for the items: 

• Students are eager to learn 

• Students make good progress 

• Students work hard 

• Students are well-behaved 



A measure which examines teachers’ perceptions of the climate in 
their schools. It consists of responses to In your school for the 
items: 

• Students are eager to learn 

• Students make good progress 

• Students work hard 

• Students are well-behaved 

This measure asks the teacher to what extent there are problems at 
the school such as 

• Student absenteeism 

• Family problems of students 

• Vandalism of school property 

• Low rates of academic achievement 

• Poor student behaviour 

• V erbal abuse of teachers 

• Lack of student interest in school 

• Lack of parental interest in school affairs 

• High staff turnover 

This measure looks at the level of perceived parental involvement 
with 

• school decision making, 

• participation in school activities such as helping with 
excursions and camps, and sport 

• parent-teacher evenings 

• general support of the school’s goals 
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Satisfaction with workload 



Satisfaction with resources 



Satisfaction with 
management 



Relationships with others 



Academic success 



Broad success 



Rating of Teachers 







This measure asks for teachers’ level of satisfaction with items such 
as 

• workload 

• out-of-class duties 

• amount of committee work 

• class sizes 

This measure looks at teacher’s satisfaction wdth 

• access to teaching resources 

• buildings and facilities 

• teaching equipment 

• grounds and sports facilities 

Asks about teacher satisfaction with 

• support from management 

• style of management 

• decision making 

Asks teachers about their perceptions of the strength of 
relationships 

• between teachers 

• between teachers and the principal 

• between teachers and parents 

Asks teachers to rate items such as How successful is your school in 

• achieving good academic results 

• teaching good communication skills 

• preparing students for higher education 

• engaging students in school life 

• helping students plan for their careers 

• providing an all-round education 

• providing a broad range of co-curricular activities 

• preparing students to be active and informed citizens 
and a rating of the school as an effective learning community. 

Asks teachers to rate items such as How successful is your school in 

• providing a curriculum that caters for all students 

• teaching skills useful in employment 

• meeting the needs of disadvantaged groups 

This group of items asks teachers to rate most teachers in their 
school on 

• mastery of their subject matter 

• ability to communicate well with students 

• being able to maintain student interest 

• managing discipline 

• respecting students as young adults 

• establishing a good learning environment. 
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Appendix 5: Multilevel models 

To examine the effects of different variables on student engagement, several multilevel models 
were tested, adding successively to each a new group or layer of variables. Initially, a variance 
components model was fitted to obtain a baseline for subsequent models. The purpose of this 
model was to estimate the proportion of variance in Engagement due to the clustering effects of 
students (level-1) within schools (level-2). Using the subscript i to refer to the student and j to 
refer to the school, the model may be written in two parts: 

a within-schools, among students part -yy = ? [Equation 1] 

and a between-school part - Poy ~ Po/ + wq,* [Equation 2] 

From Equation 1, yy (Engagement) is the dependent or response score for student i in school y. 
The intercept Poyin this within-school relationship is the average level of student’s engagement 
for school y, and ey is a random variable -- assumed to have a mean of zero - representing the sum 
of all influences on yy. The xq term in Equation 1 is a column vector of unities representing the 
constant slope (intercept) for schools. From Equation 2, the coefficient Poy is the mean 
engagement of students in the sample of schools, and uqj is a residual that varies randomly 
between schools. Since Poy may vary across schools, it is treated as a random variable at level 2. 

By combining Equations 1 and 2, a single equation version of the model can be written as 
follows: 

yy = ^Oijxo + (uoj+ ey), [Equation 3] 

where Poy^o is the fixed part of the model and the bracketed residual terms at level 2 (uqj) and 
level 1 (^y) constitute the random part of the model. Substituting the parameter estimates found 
in the variance components model into Equation 3, we can now write: 

yy (Engagement) = %jXq (-.01, .02)+ uqj(.09, .01)+ ey<.88, .01). 



These results indicate that the ratio of the parameter estimates to their standard errors for the 
school - (a^^) and student - level residual variances (Cei/) are both very large (i.e. 9 and 88 
respectively), indicating significant and stable variation at these levels. 
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